15th ICCRTS “The Evolution of C2” Development of Metrics for Trust in Automation

Research in ‘trust’ in automation has gained momentum and ‘trust’ has been identified as playing an essential role for implementing effective work-centered computer systems (Hoffman, Lee, Woods, Shadbolt, Miller & Bradshaw, 2009). In a socio-technical work system, the automation handles the majority of an algorithmically-intense workload but the human is generally a final decision-maker. Therefore, the human’s acceptance of the automation’s output is required for a successful result. Some researchers believe that system failures are connected to the human nature of trust, which is based on experiences with other humans, acting as the foundation for reliance on automated systems. However, using a common word as ‘trust’ allows for misunderstandings when used in multiple contexts. While all have some overtures of similarity, there are important unstated differences. Additionally, if trust is critical, then a method to accurately measure its goodness or level during active interaction between a human and automation would be beneficial. This paper will discuss three qualifiers for a trust evaluation such that measures can be developed to gauge a user’s trust perception over time; will lay out five components to specifically evaluate trust in automation, and propose a technique for measuring and monitoring trust in automation. Introduction For many concepts, the English language lacks exquisite distinctions in words allowing for misunderstandings in communication and causing frustrations in research as people talk past each other. The word ‘trust’ is just such a case. A simple search on Amazon.com for books that include the word ‘trust’ returns 710,654 titles as relevant. Perusing the returned list, you would note that the books run the gamut to include legal guidance (i.e., “Casenote Legal Briefs: Wills, Trusts & Estates”); art (i.e., “Trust the Process: an Artists’ Guide to Letting Go”) and religion (“Can We Trust the Gospels”). Going to SpringerLink, and performing the same simple search, 69,590 book, chapter, and journal article titles are returned with a more scientific bent but just as broad a usage ranging from computing science (“Aspects of General Security & Trust”) to medicine (“Trust, benefit, satisfaction, and burden”) to psychology (Foraging for Trust: Exploring Rationality and the Stag Hunt Game”). For comparison with another word often used in a variety of ways and contexts, in Amazon.com, 957,838 titles are returned for a ‘love’ search while in SpringerLink, only 41,687 titles are returned with a search on ‘love.’ One might expect a great number of titles for such a multi-faceted, broadly encompassing term as ‘love’ but surely most are surprised that ‘trust’ is used about as frequently, ambiguously and in so many contexts. Even in the United States’ National Security Agency’s Information Assurance Technical Framework document (2000), the term ‘trust’ is used 352 times, ranging from reference to the trustworthiness of technology, to a trusted human relationship to a Trusted Third Party. Interestingly, in the “Handbook of Trust Research,” (2006) the word ‘trust’ itself is defined in only 9 of the 22 book chapters and invariably each accepts a somewhat different definition and reference. Truly the word ‘trust’ is one of everyday parlance greatly sprinkled in everyone’s daily communications, allowing much potential confusion especially when various scientific communities research the term and try to apply their findings to critical problems. To make progress with regard to understanding and researching ‘trust,’ the concept needs to be disambiguated for particular contexts and then a method to support capturing levels or degrees of trust at a time snapshot can be developed. This paper will discuss three qualifiers (context, components and object) for a trust situation such that measures can be developed to gauge a user’s trust perception over time. Specifically focusing on trust in automation, five components were identified as relevant through a literature evaluation. An experiment was run to test the hypothesis that the five components are positively correlated to an overall evaluation of trust in the experimental condition. The experiment method and results will be described and then a technique to actively measure and monitor the trust a human has in a system is proposed. Background Trust was a topic for discussion even in Socrates’ day. Socrates never wrote his philosophies down and confined his viewpoints to spoken debate as he was concerned that portability and staleness of the written word could alter the author’s intentions especially over time. Think today of the similar potential gap between the intentions of a computer programmer or system developer and the end user. For example, consider the global positioning system (GPS). Many are the tales of the end user who over-relied on a GPS to their detriment. For example, in December 2009, an elderly couple traveling from Grants Pass, Oregon to Reno, Nevada relied on their GPS for directions and got stuck in snow for three days when their GPS unit sent them down a remote forest road. Was the shortfall in the technology developers’ viewpoint on how the technology would be used or with the user in not understanding what the system developers intended? Parasuraman and Riley (1997) discuss such types of technology usage issues as misuse, abuse, and disuse. They define use as the voluntary employment of an automation technology and discuss the factors that influence the decision to use, misuse, disuse or abuse a specific technology. Disuse is defined as the discontinuation or underutilization of technology; misuse is described as overreliance on a specific technology, and abuse is defined as inappropriate application of technology by designers or managers. (For extensive background on trust, refer to Lee and See (2004), Adams et al (2003) or Artz and Gil (2007) among many others.) Trust itself has many definitions but most have some overtures of similarity. The Mayer et al (1995) definition is the most widely accepted definition of trust, “A willingness to be vulnerable to another party when that party cannot be controlled or monitored.” If ‘agent’ is exchanged for ‘party,’ a somewhat more encompassing definition arises and this will be the general definition used for this paper. However, the definition still begs questions. Vulnerable to what extent? Vulnerable to what outcome? How willing? What are the ramifications of being vulnerable? Does the context matter? Monitored or controlled to what extent? For such a broadly used word to be the basis for evaluating the performance of socio-technical systems, the authors propose that three qualifiers are required to focus an operational definition for trust. One, that the context of interest be sufficiently defined. For example, trust in the context of corporate financial dealings would be quite different in detail from trust with respect to internet chat rooms. Also, various contexts can entail differing levels of attributes such as vulnerability, risk, and reward all of which affect levels of trust. Levels of automation should also be included in the description of context and domain of interest. Parasuraman, Sheridan, and Wickens, C. D. (2000) proposed four broad classes of functions: 1) information acquisition; 2) information analysis; 3) decision and action selection; and 4) action implementation. Within each of these types, automation can be applied across a continuum of levels from low to high, i.e., from fully manual to fully automatic. For this paper, the context is a human interacting with decision support technology where the automation does not perform until the human permits the automation. Two, that the term ‘trust’ be broken down into lower level components that allow measurement for the context of interest. This may in part address the concern in Dekker and Hollnagel (2004) that a generally used concept such as ‘trust’ be decomposed from a large construct into more measurable specifics. Additionally, decomposition can ensure a proper definition of ‘trust’ is used in the particular context of interest. For example, ‘loyalty’ was one of the keywords identified by Adams et al (2003) in their literature review but there may be disagreement on the word’s applicability for trust in automation. For this paper, the components of trust are competence, predictability, dependability, consistency and confidence, which are five attributes often cited as contributors to trust in automation. These five attributes were chosen by an in-depth literature search of trust factors. A list of all the trust factors discussed was made, and a tally of occurrences was taken. The factors that had the most repeats were used to form the above list. Competence Competence is the ability to do the task at hand. The human’s perception of automation’s competence is critical when making decisions using and trusting in technology. The ability to do the task at hand is a vital component, and the user must be aware of how to judge the competency of the automation and place appropriate use. Several researchers have cited and stated competence as a major influential factor in trust. See and Lee (2004) identified the following researchers being associated with tagging competence as a factor of trust in automation: (Barber, 1983; Butler & Cantrell, 1984; Kee & Know, 1970; Mishra, 1996). Predictability Predictability is the matching of performance with expectations. Predictability of automation plays an important role in trust. If the user can predict what the automation should do, then the user can adequately assess when the technology fails and how to perform the task without the automation. Muir (1994) discusses predictability as an important factor in a trust equation: trust = predictability + dependability + faith. Dependability Dependability, or always being there to perform, is important as the basis of trust is being vulnerable to another party (Lee & See, 2004). Rempel, et al (1985) offers dependability in automation to be essential in trust. The user must first be able to rely and depend on the automation to perform appropriately in building trust. Muir (1994) also includes dependability in the trust equation. If a user cannot depend on the automation, then the sole purpose of having automation is irrelevant and performance is ceased. Consistency Automation’s ability to be consistent in performance is imperative for a user to build trust. Consistency is being free from variation or contradiction. If the automation does not produce similar outcomes to identical tasks, the user’s trust can be skewed. Butler and Cantrell (1984) cite consistency as being one of the most influential factors of trust alongside competency. Inconsistency in automation is the first clue for a user to distrust and question the automation’s validity (Lee & See, 2004). Confidence The final component of trust is confidence which is a user attribute toward the automation. Confidence is the user’s certainty that the automation will perform appropriately. If a user has no confidence then the automation will not be used for the advantages. If the user is confident in the automation, trust can be built over time. If the user is not confident in the automation, then trust will not be able to be built over time. Going hand in hand with dependability, the user’s confidence will be crucial in relying on automation to adequately perform (Moray, Inagaki, & Itoh, 2000). If a user is too confident, he/she may abuse automation and cause damage (Lee & See, 2004). The Table 1 is the survey used in the experiment that displays how the factors were presented and asked to be rated by the participant. Table 1. Trust Factor Survey Read each item and then circle the number of the response that best describes the extent to which you would rate the Route Planner’s performance. Indicate to what extent you generally feel this way. Not At All A little Sometimes Frequently All the Time 1. To what extent is the Route Planner competent in mapping out the routes? 1 2 3 4 5 2. To what extent can the Route Planner’s routes be predicted? 1 2 3 4 5 3. To what extent can you rely on the Route Planner to plan the routes? 1 2 3 4 5 4. To what extent is the Route Planner consistent in planning the routes? 1 2 3 4 5 5. To what extent are you confident in the Route Planner’s performance? 1 2 3 4 5 The experiment described in the next section was to investigate whether these five components are reasonable for defining the qualifier of ‘lower level components’ in a trust in automation situation. The hypothesis for the experiment was that the five components listed above are positively correlated to an overall evaluation of trust in automation in the experimental condition. Three, that the object of trust be defined as research has noted the importance of the object of trust and the ability of individuals to discriminate trustworthiness of different targets. Trust is a relationship but just as love is a relationship and there is an object of love, similarly there is an object of trust. The object of trust in the experimental condition for trust in automation is a simulated global positioning system and the trust is one way. Method Participants Ninety-five undergraduate students (M = 20, SD = 3.96) from a medium Midwestern university participated in the GPS simulation experiment. A within subjects experimental design was adapted where all participants completed all the GPS conditions. Platform The automated tool used for the experiment was a Route Planner that resembles a GPS in that it assists in determining directions to a destination (figures 1-3). However, the Route Planner only displayed the entire map for an area of interest on the screen while a standard GPS could displayed either the current intersection or the entire area map. In addition, the Route Planner had the following simulated wireless updating capabilities for use in different experiments in this research: traffic jams, car accidents, burning buildings, unsafe neighborhoods, riot outbreaks, and drive by shootings. The algorithm underlying the platform provided an interesting aspect to this research. The automation determined the best course of action by first determining the next set of potential intersections from its current location. It then calculated which of those possible intersection points were closer to the final destination without considering intersections beyond that point. When the area map was complicated enough, this algorithm may have suggested the best next intersection was one which headed into a cul-de-sac or eventual dead-ends. This aspect was interesting as it can be used to stress all five factors of interest: competence, predictability, dependability, consistency and confidence. In the experimental scenarios, the user was given a navigation goal to get from point “A” on the map to point “B” within a specific constraint described below. The user had the choice to either use the automated suggested route or create a manual route. The participant would map out the entire route of his/her choice, but updated information would occur as the scenario proceeded, and the user would have the option of changing decisions based on the new information. Backtracking was not an allowed option. The following figures represent a typical decision pattern a user would go through with the Route Planner Figure 1: Representative Display Screen: Asking user if wanted suggested route displayed Figure 2: Representative Display Screen: Showing Suggested Route (Yellow): Asking user which route to select Figure 3: Representative Display Screen: Showing user choose to create a manual route (blue)